Safe driving operations of autonomous vehicles

ABSTRACT

An autonomous vehicle includes a detection system for identifying the presence of authorities or emergency vehicles on a road, such as law enforcement vehicles, fire engines, fire trucks, police officers, first responders, traffic cones, flares, etc. The detection system may include audio and visual detection system dedicated to the detection of authorities. Additionally, systems that correlate the detected authorities and/or any audible or visual instructions given by the authorities to changes in the behavior of the autonomous vehicle. The autonomous vehicle may react to the detected authorities or emergency vehicles by determining a driving related operation to safely operate around the detected authorities or emergency vehicles.

PRIORITY CLAIMS AND RELATED PATENT APPLICATIONS

This patent document claims the priority to and the benefits of U.S.Provisional Application No. 63/105,064 entitled “LEGAL AUTHORITYDETECTION AND COMPLIANCE ON A ROADWAY BY AN AUTONOMOUS VEHICLE” filed onOct. 23, 2020. The entire disclosure of the aforementioned applicationis hereby incorporated by reference as part of the disclosure of thisapplication.

TECHNICAL FIELD

The present disclosure relates generally to autonomous vehicles. Moreparticularly, the present disclosure is related to systems, apparatus,and methods for an autonomous vehicle (AV) to detect certain types ofvehicles (e.g., law enforcement vehicles), official personnel (e.g.,police officers), or stationary objects (e.g., flares or traffic conesor signage) so that the AV can operate appropriately.

BACKGROUND

One of the goals of autonomous vehicle technologies is to providevehicles that can safely navigate towards a destination with limited orno driver assistance. In some cases, an autonomous vehicle may encounterauthorities or emergency vehicles, in the form of law enforcementvehicles or official personnel. Alternatively, or additionally, localauthorities may post signs, enable flashing lights, place flares, orotherwise visually indicate a deviation from normal traffic patterns ina section of roadway. Currently in such cases, an assistance of a driveris usually needed to manually drive the autonomous vehicle bypassing theunexpected situation. Previous systems do not currently enableautonomous vehicles to adequately detect authorities and comply withinstructions or legal requirements without human intervention.

SUMMARY

Detection of certain types of vehicles (e.g., law enforcement and/orfirst responder vehicles), official personnel, audible instructions,and/or visual indicators can allow an autonomous vehicle to safelyoperate by complying with applicable laws. Such detection can alsoenable the autonomous vehicle to operate safely around persons and/orproperty surrounding the autonomous vehicle. Systems, apparatuses, andmethods are described herein that allow an autonomous vehicle to detectthe presence of authorities (e.g., certain type of vehicles and/orofficial personnel) and/or to perform driving related operations tocomply with instructions and/or local laws for safe and lawfuloperation.

In an example embodiment, a method of autonomous driving operationincludes determining, by a computer located in an autonomous vehicle,that an emergency vehicle is located on a road on which the autonomousvehicle is being driven; obtaining, in response to the determining, adriving related operation of the autonomous vehicle to operate aroundthe emergency vehicle; and causing the autonomous vehicle to driveaccording to the driving related operation by sending instructionsrelated to the driving related operation to one or more devices in theautonomous vehicle.

In some embodiments, the obtaining the driving related operation and thecausing the autonomous vehicle to drive according to the driving relatedoperation is performed in response to: (1) the determining that theemergency vehicle is located on the road, and (2) determining that theemergency vehicle is located within a pre-determined distance of alocation of the autonomous vehicle. In some embodiments, the obtainingthe driving related operation and the causing the autonomous vehicle todrive according to the driving related operation is performed inresponse to: (1) the determining that the emergency vehicle is locatedon the road, (2) determining that the emergency vehicle is locatedwithin a pre-determined distance of a location of the autonomousvehicle, and (3) determining that the emergency vehicle is operating asiren. In some embodiments, the obtaining the driving related operationincludes: performing a first determination that a current speed of theautonomous vehicle is greater than a threshold value, and performing asecond determination, in response to the first determination, of anamount of braking force to apply that reduces the current speed of theautonomous vehicle to less than or equal to the threshold value; and theinstructions causing the autonomous vehicle to drive according to thedriving related operation include a first instruction that cause theautonomous vehicle to apply the amount of braking force.

In some embodiments, the second determination is performed in responseto the first determination and in response to a third determination thatthe autonomous vehicle is driving on a lane on the road immediatelyadjacent to an area or another lane where the emergency vehicle islocated. In some embodiments, the obtaining the driving relatedoperation includes: performing a first determination that the autonomousvehicle is driving on the road on a first lane having one side that isimmediately adjacent to an area or another lane where the emergencyvehicle is located, performing a second determination that the roadincludes a second lane immediately adjacent to the first lane and onanother side opposite to the one side, and performing a thirddetermination, in response to the first determination and the seconddetermination, of an amount of steering to apply to steer the autonomousvehicle from the first lane to the second lane; the instructions causingthe autonomous vehicle to drive according to the driving relatedoperation include a first instruction that cause the autonomous vehicleto apply the amount of steering to steer the autonomous vehicle from thefirst lane to the second lane.

In some embodiments, a system for autonomous vehicle operation includesan autonomous vehicle comprising a computer that includes a processorconfigured to implement a method, comprising: determine that anemergency vehicle is located on a road on which the autonomous vehicleis being driven; obtain, in response to the determine, a driving relatedoperation of the autonomous vehicle to operate around the emergencyvehicle; and cause the autonomous vehicle to drive according to thedriving related operation by the processor configured to sendinstructions related to the driving related operation to one or moredevices in the autonomous vehicle.

In some embodiments, the processor of the computer is further configuredto: send, to a second computer located outside of the autonomousvehicle, information related to the emergency vehicle, where theinformation includes a position information that indicates a position ofthe emergency vehicle, and where the information indicates that theemergency vehicle is located within a first pre-determined distance of alocation of the autonomous vehicle; and where the second computercomprises a second processor configured to implement a second method,comprising: receive the information related to the emergency vehicle; inresponse to a determination that the second autonomous vehicle islocated within a second pre-determined distance of a position of theemergency vehicle and that the second autonomous vehicle is drivingtowards the emergency vehicle: send the information related to theemergency vehicle to a second autonomous vehicle, where the secondpre-determined distance is greater than or equal to the firstpre-determined distance.

In some embodiments, the obtain the driving related operation isperformed by the processor configured to: receive a traffic informationof an area towards which the autonomous vehicle is driving; determine,based on the traffic information, that the area has a high trafficlevel; determine an alternate route to drive the autonomous vehiclearound the area having the high traffic level; and determine theinstructions for any one or more of a steering system, brakes, ortransmission to cause the autonomous vehicle to drive on the alternateroute. In some embodiments, the instructions for the driving relatedoperation include a first instruction to apply brakes and a secondinstruction steer the autonomous vehicle from a lane on the road onwhich the autonomous vehicle is being driven and to an area along a sideof the road. In some embodiments, the emergency vehicle includes a lawenforcement vehicle, an ambulance, a fire engine, or a fire truck. Insome embodiments, the emergency vehicle includes a stationary objectthat includes a traffic cone, a flare, or a traffic related signage.

In some embodiments, the obtaining the driving related operation and thecausing the autonomous vehicle to drive according to the driving relatedoperation is performed in response to: (1) determining that theemergency vehicle is driving towards the autonomous vehicle, (2)determining that the emergency vehicle is located within apre-determined distance of a location of the autonomous vehicle, and (3)determining that the emergency vehicle is operating a siren. In someembodiments, the emergency vehicle is determined to be driving towardsthe autonomous vehicle upon determining that a first number of pixelsassociated with the emergency vehicle in a first image is less than asecond number of pixels associated with the emergency vehicle in asecond image, the first image and the second image are obtained by acamera on the autonomous vehicle, and the first image is obtained at afirst time that is earlier in time than a second time when the secondimage is obtained. In some embodiments, the emergency vehicle isdetermined to be driving towards the autonomous vehicle upon determiningthat a first sound level associated with the siren is less than a secondsound level associated with the siren, the first sound level and thesecond sound level are obtained by a microphone on the autonomousvehicle, and the first sound level is obtained at a first time that isearlier in time than a second time when the second sound level isobtained.

In some embodiments, the driving related operation is obtained based ona location of the autonomous vehicle and based on a rule associated withthe emergency vehicle of a state or of an area in which the autonomousvehicle is located. In some embodiments, the driving related operationis obtained using a pre-configured set of rules stored in the computerfor a plurality of states or areas, each rule is associated with atleast one state or at least one area and a corresponding driving relatedrule associated with at least one emergency vehicle, and the pluralityof states or areas include the state or the area in which the autonomousvehicle is located. In some embodiments, the method further includessending, to a second computer located outside of the autonomous vehicle,information related to the emergency vehicle; and where the obtainingthe driving related operation includes receiving, from the secondcomputer, the instructions related to the driving related operation. Insome embodiments, the information is sent to the second computer inresponse to determining that one or more attributes of the emergencyvehicle cannot be determined by the computer in the autonomous vehicle.

In yet another exemplary aspect, the above-described method is embodiedin a non-transitory computer readable storage medium comprising codethat when executed by a processor, causes the processor to perform themethods described in this patent document.

In yet another exemplary embodiment, a device that is configured oroperable to perform the above-described methods is disclosed. In yetanother exemplary embodiment, a system that includes a computer with aprocessor configured or operable to perform the above-described methodsis disclosed.

The above and other aspects and their implementations are described ingreater detail in the drawings, the descriptions, and the claims.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 illustrates a schematic diagram of a system including anautonomous vehicle.

FIG. 2 is an exemplary method for detecting authorities and operating anautonomous vehicle in compliance with instructions and legalrequirements.

FIG. 3 illustrates a system that includes one or more autonomousvehicles, a control center or oversight system with a human operatorand/or an interface for third-party interaction.

FIG. 4 illustrates an example flowchart of operations performed by anautonomous vehicle.

DETAILED DESCRIPTION

Vehicles traversing highways and roadways are legally required to complywith authorities or emergency vehicles to safely operate the vehicles.For autonomous vehicles, such as autonomous tractor trailer vehicles,detection of authorities or emergency vehicles such as emergencyvehicles, official personnel and/or visual instructions (e.g., signs) onthe road is essential for lawful and safe operation of the vehicle. Thispatent document describes systems, apparatuses and methods for safe andlawful operation of an autonomous vehicle on a roadway, includingdetection of a presence of authorities and/or visual instructions, andexecution of driving related operations so that the autonomous vehiclecan comply with the law.

Section I of this patent document provides an overview of the devicesand/or systems located on or in an autonomous vehicle, such as anautonomous tractor trailer vehicle. Section II of this patent documentdescribes techniques to detect a presence of an authority on a roadand/or to perform driving related operations in response to detectingthe presence of the authority on the road. In this patent document, theterm “authority” or “emergency vehicle” may include law enforcementvehicles, fire engines/trucks, ambulance, military vehicles, or mayinclude stationary objects such as traffic cones, flares, signage, etc.Section III of this patent document describes an oversight system withwhich one or more autonomous vehicles can communicate regarding thepresence of the authority to facilitate appropriate driving related Theexample headings for the various sections below are used to facilitatethe understanding of the disclosed subject matter and do not limit thescope of the claimed subject matter in any way. Accordingly, one or morefeatures of one example section can be combined with one or morefeatures of another example section.

I. Autonomous Vehicle Ecosystem

FIG. 1 shows a system 100 that includes an autonomous vehicle 105, suchas an autonomous tractor trailer vehicle. The autonomous vehicle 105includes a plurality of vehicle subsystems 140 and an in-vehicle controlcomputer 150 The plurality of vehicle subsystems 140 includes vehicledrive subsystems 142, vehicle sensor subsystems 144, and vehicle controlsubsystems. An engine or motor, wheels and tires, a transmission, anelectrical subsystem, and a power subsystem may be included in thevehicle drive subsystems. The engine of the autonomous vehicle may be aninternal combustion engine, a fuel-cell powered electric engine, abattery powered electrical engine, a hybrid engine, or any other type ofengine capable of moving the wheels on which the autonomous vehicle 105moves. The autonomous vehicle 105 have multiple motors or actuators todrive the wheels of the vehicle, such that the vehicle drive subsystems142 include two or more electrically driven motors. The transmission mayinclude a continuous variable transmission or a set number of gears thattranslate the power created by the engine into a force that drives thewheels of the vehicle. The vehicle drive subsystems may include anelectrical system that monitors and controls the distribution ofelectrical current to components within the system, including pumps,fans, and actuators. The power subsystem of the vehicle drive subsystemmay include components that regulate the power source of the vehicle.

Vehicle sensor subsystems 144 can include sensors for general operationof the autonomous vehicle 105 and an authority detection subsystem 145.The sensors for general operation of the autonomous vehicle may includecameras, a temperature sensor, a inertial sensor (IMU), a globalpositioning system, a light sensor, a LIDAR system, a radar system, andwireless communications.

An audio detector, such as a microphone or array of microphones, may beincluded in the authority detection subsystem 145, as may rear-facingcameras. The audio detector, e.g., microphone(s), of the authoritydetection subsystem 145 are configured to receive audio indications ofthe presence of, or instructions from, authorities, including sirens andcommands such as “pull over.” One or more microphones of the audiodetector are mounted or located on the external portion of the vehicle,such as on the outside of the tractor portion of an autonomous vehicle105. Microphones used in the authority detection subsystem 145 may beany suitable type, mounted such that they are effective both when theautonomous vehicle 105 is at rest, as well as when it is moving atnormal driving speeds. The speeds at which the microphone may detectsirens or audio commands from authorities may range from 0 to 50 milesper hour or more. In some configurations, the authority detectionsubsystem 145 includes microphones that are configured to detect thedirection in which the origin of a siren or other intense sound ismoving. Microphones may be located on both the driver's side and thepassenger's side, or the right and left sides, of the autonomous vehiclebody to facilitate the ability of the authority detection subsystem 145to determine the movement of source of an intense or persistent soundsuch as a siren wail. As further explained in this patent document, whenthe audio detector receives an audio indication (e.g., digitized soundrecording) of the presence of, or instructions from, authorities, theaudio detector sends the received audio indication to the legalauthority detection module 165 so that the legal authority detectionmodule 165 can process the audio indication to determine a presence ofan authority on the road.

Cameras that are part of the authority detection subsystem 145 may berear-facing so that flashing lights from emergency vehicles may beobserved from all around the autonomous vehicle 105. These cameras mayinclude video cameras, cameras with filters for specific wavelengths, aswell as any other cameras suitable to detect emergency vehicle lightsbased on color, flashing, of both color and flashing. The authoritydetection subsystem 145 may also use forward facing or side facingcameras from the vehicle sensor subsystem 144. Video data obtained bythe authority detection subsystem 145 is sent to the legal authoritydetection module 165 so that the legal authority detection module 165can process the video data to determine a presence of an authority onthe road.

The vehicle control subsystem 146 may be configured to control operationof the autonomous vehicle, or truck, 105 and its components.Accordingly, the vehicle control subsystem 146 may include variouselements such as an engine power output subsystem, a brake unit, anavigation unit, a steering system, and an autonomous control unit. Theengine power output may control the operation of the engine, includingthe torque produced or horsepower provided, as well as provide controlthe gear selection of the transmission. The brake unit can include anycombination of mechanisms configured to decelerate the autonomousvehicle 105. The brake unit can use friction to slow the wheels in astandard manner. The brake unit may include an Anti-lock brake system(ABS) that can prevent the brakes from locking up when the brakes areapplied. The navigation unit may be any system configured to determine adriving path or route for the autonomous vehicle 105. The navigationunit may additionally be configured to update the driving pathdynamically while the autonomous vehicle 105 is in operation. In someembodiments, the navigation unit may be configured to incorporate datafrom the GPS device and one or more predetermined maps so as todetermine the driving path for the autonomous vehicle 105. The steeringsystem may represent any combination of mechanisms that may be operableto adjust the heading of autonomous vehicle 105 in an autonomous mode orin a driver-controlled mode. As further explained in this patentdocument, the devices of the vehicle control subsystem 146 may becontrolled by the compliance module 166 and/or driving operation module168 to perform driving related operations on the autonomous vehicle 105.

The autonomous control unit may represent a control system configured toidentify, evaluate, and avoid or otherwise negotiate potential obstaclesin the environment of the autonomous vehicle 105. In general, theautonomous control unit may be configured to control the autonomousvehicle 105 for operation without a driver or to provide driverassistance in controlling the autonomous vehicle 105. In someembodiments, the autonomous control unit may be configured toincorporate data from the GPS device, the RADAR, the LIDAR, the cameras,and/or other vehicle subsystems to determine the driving path ortrajectory for the autonomous vehicle 105.

An in-vehicle control computer 150, which may referred to as a VCU,includes a vehicle subsystem interface 160, a driving operation module168, one or more processors 170, a legal authority detection module 165,a compliance module 166, a memory 175, and a network communicationssubsystem 178. This in-vehicle control computer 150 controls many, ifnot all, of the operations of the autonomous vehicle 105 in response toinformation from the various vehicle subsystems 140 as explained inSection II of this patent document and/or in response to communicatingwith an oversight system as explained in Section III of this patentdocument. The one or more processors 170 execute the operationsassociated with the legal authority detection module 165 that allow thesystem to determine that an authority, such as a law enforcement oremergency service vehicle or associated personnel, are nearby and/orgiving instructions. Data from the authority detection subsystem 145 isprovided to the legal authority detection module 165 so that thedetermination of a presence of an authority can be made. The compliancemodule 166 receives from the legal authority detection module 165 anindication that a presence of an authority on the road been determined.In response to the indication, the compliance module 166 can determinethe course of action (e.g., driving related operation(s) such assteering, braking, etc.) that should be performed by the autonomousvehicle 105. Data from other vehicle sensor subsystems 144 may beprovided to the compliance module so that the course of action may beappropriately determined. Alternatively, or additionally, the compliancemodule 166 may determine the course of action in conjunction withanother operational or control module, such as the driving operationmodule 168.

The memory 175 may contain additional instructions as well, includinginstructions to transmit data to, receive data from, interact with, orcontrol one or more of the vehicle drive subsystem 142, the vehiclesensor subsystem 144, and the vehicle control subsystem 146. Thein-vehicle control computer (VCU) 150 may, via the driving operationmodule 168, control the function of the autonomous vehicle 105 based oninputs received from various vehicle subsystems (e.g., the vehicle drivesubsystem 142, the vehicle sensor subsystem 144, and the vehicle controlsubsystem 146). Additionally, driving operation module 168 may sendinformation to the VCU 150 so that the VCU 150 may send information tothe vehicle control subsystems 146 to direct the trajectory, velocity,signaling behaviors, and the like, of the autonomous vehicle 105. Theautonomous control vehicle control subsystem may receive a course ofaction to be taken from the compliance module 166 and/or the drivingoperation module 168 of the VCU 150 and consequently relay instructionsto other subsystems to execute the course of action.

II. Legal Authority Detection and Autonomous Driving ComplianceTechniques

FIG. 2 shows a flow diagram for the detection of legal authorities andcompliance with any laws or explicit instructions given. The authoritydetection subsystem 145 receives visual, auditory, or both visual andauditory signals via cameras or microphones at operation 205. The one ormore microphone and/or one or more cameras associated with the authoritydetection subsystem 145 can provide digital auditory (or audio) dataand/or digital visual (or video) data. At operation 210, the digitalvisual and/or auditory data is/are transmitted from the authoritydetection subsystem 145 to the in-vehicle control computer system (VCU)150 for signal processing.

At operation 215, the legal authority detection module receives thedigital visual and/or auditory data transmitted from the authoritydetection system. At operation 220, the legal authority detection moduleperforms signal processing on the digital auditory and/or visual data todetermine whether one or more authorities are located on the road withinan area of a location of the autonomous vehicle. In some embodiments,the legal authority detection module can process the digital visual datareceived from one or more cameras on the autonomous vehicle to determinea presence of a law enforcement vehicle, fire truck or ambulance in theimages of the digital visual data. In some embodiments, the legalauthority detection module can determine, using digital auditory and/orvisual data received over time, that a law enforcement vehicle, firetruck or ambulance in the road is driving towards the autonomous vehicleor driving away from the vehicle. In such embodiments, the legalauthority detection module can first determine a presence of anauthority (e.g., ambulance) using digital auditory and/or visual dataobtained from the authority detection subsystems at a first time. Thelegal authority detection module can also determine a first number ofpixels associated with the authority (e.g., ambulance) and/or a firstsound level (e.g., in decibels) of the siren operated by the authority(e.g., ambulance) from the digital visual data and/or the digitalauditory data obtained at the first time, respectively.

Next, the legal authority detection module can determine from anotherdigital auditory and/or visual data obtained from the authoritydetection subsystems at a second time a second number of pixelsassociated with the authority and/or a second sound level of the sirenassociated with the authority, where the second time is later in timethan the first time. In this embodiment, the legal authority detectionmodule can determine that the authority is driving towards theautonomous vehicle upon determining that the second number of pixels isgreater than the first number of pixels and/or that the second soundlevel is greater than the first sound level. The legal authoritydetection module can determine that the authority is driving away fromthe autonomous vehicle by determining that the second number of pixelsis less than the first number of pixels and/or that the second soundlevel is less than the first sound level. In some embodiments, the legalauthority detection module can use image processing techniques todetermine one or more of the following information about the authority:identity (e.g., law enforcement vehicle) of the authority, the locationof the authority, and/or the speed of the authority (if applicable),direction in which the authority if traveling (if applicable, e.g.,towards or away from the autonomous vehicle), and/or whether theauthority is operating its siren.

In some embodiments, the legal authority detection module can determinethe direction in which the authority is driving towards, away from, oraround the autonomous vehicle by identifying the cameras that obtainedthe video data comprising the authority and by determining thedirection(s) in which the cameras are facing. In some embodiments, thelegal authority detection module can determine the direction in whichthe authority is driving towards or around the autonomous vehicle bycomparing sound levels of a siren captured by multiple microphones thatmay be located at several places on the autonomous vehicle. In someembodiments, the legal authority detection module can use the digitalvisual data obtained from one or more cameras to determine that theauthority (e.g., law enforcement vehicle or first responder vehicle) ison a road that is separate from or is same as the road on which theautonomous vehicle is operating.

In an example implementation, the legal authority detection module cancompare the digital auditory data with a previously stored audio data ofa siren in the VCU to determine whether a law enforcement vehicle, firetruck and/or ambulance operating a siren is/are driving or located closeto the autonomous vehicle. In another example implementation, the legalauthority detection module can compare the digital visual data withpreviously stored images of a law enforcement vehicle, fire truck and/orambulance in the VCU to determine whether a law enforcement vehicle,fire truck and/or ambulance is/are driving or located close to theautonomous vehicle.

In some embodiments, the legal authority detection module can determine,using digital visual data, a presence of a visual instruction fromauthorities such as flares, cones, or signage. In some embodiments, thelegal authority detection module can determine, using digital visualdata, that an official is present on a road (e.g., a police officer iswalking next to the road or a construction worker is directing traffic,etc.). In some embodiments, the legal authority detection module canprocess the digital auditory data received from a microphone todetermine an absence of a siren sound that is associated with a lawenforcement vehicle, fire truck or ambulance. In some embodiments, thelegal authority detection module can process the digital visual datareceived from one or more cameras on the autonomous vehicle to determinean absence of a law enforcement vehicle, fire truck or ambulance in theimages of the digital visual data.

At operation 225, the legal operation detection module can send aninformation to the compliance module indicating a presence (or absence)or one or more authorities in an area where the autonomous vehicle isoperating. In some embodiments, the information about the authority sentby the legal authority detection module to the compliance module caninclude an indication that the authority is present in the area wherethe autonomous vehicle is operating, an identification of the authority(e.g., law enforcement vehicle), the location of the authority, thespeed of the authority (if applicable), the direction in which theauthority is traveling (if applicable, e.g., towards or away from theautonomous vehicle), and/or whether the authority is operating itssiren. Based on the information about the authority received from thelegal authority detection module, at operation 225, the compliancemodule can determine a driving related operation (e.g., steering,braking, changing lanes, stopping, etc.) for the autonomous vehicle toperform as further explained below.

For example, when the compliance module determines that an authoritydriving towards the autonomous vehicle and/or is located within apre-determined distance of the autonomous vehicle and/or the authorityis operating its siren, then the compliance module can determine andsend instructions for driving related operation (at operation 230) tothe steering system to the brakes to steer the autonomous vehicle and toapply brakes to stop the autonomous vehicle along the side of the road.In another example, if the compliance module determines that anauthority is stopped or parked next to a road having one or more lanes(or on a shoulder of the road), and if the compliance module determinesthat the autonomous vehicle is driving at a speed higher than athreshold (e.g., pre-determined) value, then the compliance module candetermine an amount of braking force to apply to reduce the autonomousvehicle's speed to the threshold value or to a speed less than thethreshold value. In this example, the compliance module can sendinstructions for a driving related operation to apply the amount ofbraking force (e.g., braking pressure, frictional force applied to slowwheels, clamping force exerted by calipers) to lower the speed to thethreshold value or to another speed less than the threshold value todrive safely next to the stopped or parked authority. In this example,the compliance module can send instructions for a driving relatedoperation (at operation 230) to apply the amount of braking force tolower the speed to the threshold value upon determining that theautonomous vehicle's speed is greater than a threshold value, and upondetermining that the autonomous vehicle is driving on the right-mostlane or the left-most lane that is immediately adjacent to an area orlocation or another lane on the road where the authority is stopped orparked.

In yet another example, if the compliance module determines that anauthority is stopped or parked next to a road having multiple lanes (oron a shoulder of the road), and if the compliance module determines thatthe autonomous vehicle is driving on a first lane (e.g., the right-mostlane or the left-most lane) that is immediately adjacent to the locationon the road where the authority is stopped or parked, then thecompliance module can send instructions for a driving related operation(at operation 230) to the steering system to change lanes from the firstlane to the second lane so that there is at least one lane in betweenthe lane where the autonomous vehicle is driving and the stopped orparked authority. In this example, the compliance module can determinean amount of steering needed to steer the autonomous vehicle from thefirst lane to the second lane.

In yet another example, the compliance module can receive trafficinformation of an area towards which the autonomous vehicle is driving,where the traffic information indicates a level of traffic (e.g., high,medium, low) of the area. In this example, if the compliance moduledetermines that the authority is stopped or parked in that area and ifthe compliance module determines that the traffic information indicatesthat traffic in that area is high, then the compliance module candetermine whether an alternate route (e.g., route other than one onwhich the autonomous vehicle is driving) exists to reroute theautonomous vehicle. In this example, if the compliance module determinesthat the autonomous vehicle can be rerouted to the alternate routearound the high traffic area to its destination, then the compliancemodule can determine and send to the steering system, brakes,transmission, and/or other devices in the autonomous vehicleinstructions related to the driving related operations to drive theautonomous vehicle on the alternate route.

In some embodiments, the compliance module can determine the drivingrelated operations as explained above based on the law of the state orcountry or area in which the autonomous vehicle is driving. The VCU canstore a look-up table that includes a pre-configured set of rulesapplicable to authorities for various states or areas in which theautonomous vehicle is expected to be driven. Each rule in thepre-configured set of rules may include an identification of a state orarea and a driving related rule associated with at least one authorityor emergency vehicle on the road for that state or area. For example,one entry in the look-up table may indicate an identification of a firststate or a first area and a description of an associated driving relatedrule that states that there must be at least one lane in between thelocation of the autonomous vehicle and the authority when the authorityis parked or stopped next to the road with its lights flashing. Inanother example, another entry in the look-up table may indicate anidentification of a second state or a second area and a description ofan associated driving related rule that states that a vehicle's speedmust be at or below a threshold value if an authority is parked orstopped next to the road on which vehicles are driving or if anauthority or emergency vehicle (e.g., construction signage) indicatesthat construction is being performed on the road. The compliance modulecan access the look-up table using the location of the autonomousvehicle (e.g., obtained from a GPS device on or in the autonomousvehicle) to determine relevant rules or laws of the state or area inwhich the autonomous vehicle is driving. In some embodiments, thelook-up table may include multiple rules associated with a state or anarea. For example, the look-up table may indicate that in response to anauthority located on the road with its lights flashing and/or its sirenoperating, a vehicle is expected either to move to another lane to keepa one-lane distance between the vehicle and the authority if the vehicleis in a lane adjacent to the authority or the vehicle is expected toslow down to a pre-determined speed if the vehicle cannot move over tothe another lane. Based on the rules or laws of the area where theautonomous vehicle is driving, the compliance module can send drivingrelated operations to the autonomous vehicle to comply with local orstate rules or law.

In some embodiments, the compliance module can send information relatedto the driving related operations to be displayed on a display monitorin the autonomous vehicle to inform a human driver in the autonomousvehicle.

At operation 230, the driving related operations determined by thecompliance module can be transmitted from the VCU 150 to one or moredevices in the vehicle control subsystems in 230 and/or to other devices(e.g., transmission in vehicle drive subsystem 142) in the autonomousvehicle. At operation 235, the devices of the vehicle control subsystem146 and/or the vehicle drive subsystem 142 can perform driving relatedoperations based on the instructions provided by the compliance moduleas explained above. The vehicle control subsystems and/or the vehicledrive subsystems then cause the autonomous vehicle 105 to operate inaccordance with the driving related operations received from the VCU150.

III. Oversight System

FIG. 3 illustrates a system 300 that includes one or more autonomousvehicles 105, a control center or oversight system 350 with a humanoperator 355, and an interface 362 for third-party personnel 360interaction. A human operator 355 may also be known as a remoter centeroperator (RCO). Communications between the autonomous vehicles 105,oversight system 350 and user interface 362 take place over a network370. In some instances, where not all the autonomous vehicles 105 in afleet are able to communicate with the oversight system 350, theautonomous vehicles 105 may communicate with each other over the network370 or directly. As described with respect to FIG. 1, the VCU 150 ofeach autonomous vehicle 105 may include a module for networkcommunications 178.

An autonomous vehicle may be in communication with an oversight system.The oversight system may include one or more computers or servers thatmay serve many purposes, including: tracking the progress of one or moreautonomous vehicles (e.g., an autonomous vehicle); tracking the progressof a fleet of autonomous vehicles; sending maneuvering instructions toone or more autonomous vehicles; monitoring the health of the autonomousvehicle(s); monitoring the status of the cargo of each autonomousvehicle in contact with the oversight system; facilitate communicationsbetween third parties (e.g., law enforcement, clients whose cargo isbeing carried) and each, or a specific, autonomous vehicle; allow fortracking of specific autonomous vehicles in communication with theoversight system (e.g., third-party tracking of a subset of vehicles ina fleet); arranging maintenance service for the autonomous vehicles(e.g., oil changing, fueling, maintaining the levels of other fluids);alerting an affected autonomous vehicle of changes in traffic or weatherthat may adversely impact a route or delivery plan; pushing over the airupdates to autonomous vehicles to keep all components up to date; andother purposes or functions that improve the safety for the autonomousvehicle, its cargo, and its surroundings. An oversight system may alsodetermine performance parameters of an autonomous vehicle or autonomousvehicle, including any of: data logging frequency, compression rate,location, data type; communication prioritization; how frequently toservice the autonomous vehicle (e.g., how many miles between services);when to perform a minimal risk condition (MRC) maneuver while monitoringthe vehicle's progress during the maneuver; when to hand over control ofthe autonomous vehicle to a human driver (e.g., at a destination yard);ensuring an autonomous vehicle passes pre-trip inspection; ensuring anautonomous vehicle performs or conforms to legal requirements atcheckpoints and weight stations; ensuring an autonomous vehicle performsor conforms to instructions from a human at the site of a roadblock,cross-walk, intersection, construction, or accident; and the like.

A compliance module of an autonomous vehicle can send to an oversightsystem the following information as explained in Section II of thispatent document: information about the authority received from legalauthority detection module, and/or driving related operations determinedby the compliance module. In some embodiments, the oversight system 350or the interface 362 can display on a monitor that an authority has beendetected in response to which the human operator 355 or a third partypersonnel 360 can send via the oversight system 350 or the interface 362instructions for driving related operations to the compliance module tosend to the vehicle control subsystem 146 and/or vehicle drive subsystem142. In some embodiments, if the oversight system 350 or interface 362displays on a monitor the driving related operations determined by thecompliance module, a person may override or adjust the driving relatedoperations. In such embodiments, for example, the human operator 355 orthe third party personnel 360 may, via the oversight system 350 or theinterface 362, send instructions to the compliance module to sendinstructions to the vehicle control subsystem 146 to lower theautonomous vehicle's speed to a certain value lower than thepredetermined value determined by the compliance module. In anotherexample, the human operator 355 or third party personnel 360 may via theoversight system 350 or the interface 362 send instruction to thecompliance module to send instructions to the vehicle control subsystem146 to exit a highway or freeway.

To allow for communication between autonomous vehicles in a fleet and anoversight system or command center, each autonomous vehicle may beequipped with a communication gateway. The communication gateway mayhave the ability to do any of the following: allow for AV to oversightsystem communication (i.e. V2C) and the oversight system to AVcommunication (C2V); allow for AV to AV communication within the fleet(V2V); transmit the availability or status of the communication gateway;acknowledge received communications; ensure security around remotecommands between the AV and the oversight system; convey the AV'slocation reliably at set time intervals; enable the oversight system toping the AV for location and vehicle health status; allow for streamingof various sensor data directly to the command or oversight system;allow for automated alerts between the AV and oversight system; complyto ISO 21434 standards; and the like.

An oversight system or command center may be operated by one or morehuman, also known as an operator or a remote center operator (RCO). Theoperator may set thresholds for autonomous vehicle health parameters, sothat when an autonomous vehicle meets or exceeds the threshold,precautionary action may be taken. Examples of vehicle health parametersfor which thresholds may be established by an operator may include anyof: fuel levels; oil levels; miles traveled since last maintenance; lowtire-pressure detected; cleaning fluid levels; brake fluid levels;responsiveness of steering and braking subsystems; Diesel exhaust fluid(DEF) level; communication ability (e.g., lack of responsiveness);positioning sensors ability (e.g., GPS, IMU malfunction); impactdetection (e.g., vehicle collision); perception sensor ability (e.g.,camera, LIDAR, radar, microphone array malfunction); computing resourcesability (e.g., VCU or ECU malfunction or lack of responsiveness,temperature abnormalities in computing units); angle between a tractorand trailer in a towing situation (e.g., tractor-trailer, 18-wheeler, orsemi-truck); unauthorized access by a living entity (e.g., a person oran animal) to the interior of an autonomous vehicle; and the like. Theprecautionary action may include execution of a minimal risk condition(MRC) maneuver, seeking service, or exiting a highway or other suchre-routing that may be less taxing on the autonomous vehicle. Anautonomous vehicle whose system health data meets or exceeds a thresholdset at the oversight system or by the operator may receive instructionsthat are automatically sent from the oversight system to perform theprecautionary action.

The operator may be made aware of situations affecting one or moreautonomous vehicles in communication with or being monitored by theoversight system that the affected autonomous vehicle(s) may not beaware of. Such situations may include: irregular or sudden changes intraffic flow (e.g., traffic jam or accident); abrupt weather changes;abrupt changes in visibility; emergency conditions (e.g., fire,sink-hole, bridge failure); power outage affecting signal lights;unexpected road work; large or ambiguous road debris (e.g., objectunidentifiable by the autonomous vehicle); law enforcement activity onthe roadway (e.g., car chase or road clearing activity); and the like.These types of situations that may not be detectable by an autonomousvehicle may be brought to the attention of the oversight system operatorthrough traffic reports, law enforcement communications, data from othervehicles that are in communication with the oversight system, reportsfrom drivers of other vehicles in the area, and similar distributedinformation venues. An autonomous vehicle may not be able to detect suchsituations because of limitations of sensor systems or lack of access tothe information distribution means (e.g., no direct communication withweather agency). An operator at the oversight system may push suchinformation to affected autonomous vehicles that are in communicationwith the oversight system. The affected autonomous vehicles may proceedto alter their route, trajectory, or speed in response to theinformation pushed from the oversight system. In some instances, theinformation received by the oversight system may trigger a thresholdcondition indicating that MRC (minimal risk condition) maneuvers arewarranted; alternatively, or additionally, an operator may evaluate asituation and determine that an affected autonomous vehicle shouldperform a MRC maneuver and subsequently send such instructions to theaffected vehicle. In these cases, each autonomous vehicle receivingeither information or instructions from the oversight system or theoversight system operator uses its on-board computing unit (i.e. VCU) todetermine how to safely proceed, including performing a MRC maneuverthat includes pulling-over or stopping.

Other interactions that the remote center operator (RCO) may have withan autonomous vehicle or a fleet of autonomous vehicle includes any ofthe following. A computer of the oversight system 350 may receive from afirst autonomous vehicle an indication of a presence of an authoritywithin a first pre-determined distance of the location of the firstautonomous vehicle and/or relevant information about the authority(e.g., position information of the authority).

The computer of the oversight system 350 may also obtain positioninformation of one or more autonomous vehicle that may be monitored bythe computer of the oversight system 350. Using a location informationof a second autonomous vehicle and the position information of theauthority, the computer of the oversight system 350 may determine thatthe second autonomous vehicle is located within a second pre-determineddistance of the location of the authority. The computer of the oversightsystem 350 may send information about the authority received from theautonomous vehicle to the second autonomous vehicle upon determiningthat the second autonomous vehicle is driving towards the authority andis within the second pre-determined distance of a location of theauthority, where the second pre-determined distance is greater than orequal to the first pre-determined distance. The information about theauthority provided to the second autonomous vehicle can enable thecompliance module of the second autonomous vehicle to determine drivingrelated operations based on the presence of the authority at thelocation. In some embodiments, a human operator 355 or a third-partypersonnel 360 can via the oversight system 350 or the interface 362 sendinformation to the second autonomous vehicle to perform certain drivingrelated operations (e.g., reroute the second autonomous vehicle or exitthe highway, etc.).

In some embodiments, the information about the authority may be sent bythe compliance module of an autonomous vehicle upon determining thatcertain information about the authority cannot be adequately determined.For example, if the legal authority detection module cannot determineinformation conveyed by a traffic sign posted on a road or whether avehicle belongs to law enforcement, the legal authority detection moduleindicate a lack of determination of an attribute of the authority to thecompliance module and the compliance module can send information aboutthe authority (e.g., images) to the oversight system for a human toassist in determining the driving related operation.

An oversight system or command center may allow a third party tointeract with the oversight system operator, with an autonomous vehicle,or with both the human system operator and an autonomous vehicle. Athird party may be a customer whose goods are being transported, a lawenforcement or emergency services provider, or a person assisting theautonomous vehicle when service is needed.

Actions that an autonomous vehicle, particularly an autonomous vehicle,as described herein may be configured to execute to safely traverse acourse while abiding by the applicable rules, laws, and regulations mayinclude those actions successfully accomplished by an autonomous vehicledriven by a human. These actions, or maneuvers, may be described asfeatures of the truck, in that these actions may be executableprogramming stored on the VCU 150 (i.e., the in-vehicle control computerunit). These actions or features may include those related to reactionsto the detection of certain types of conditions or objects such as:appropriate motion in response to detection of an emergency vehicle withflashing lights; appropriate motion in response to detecting one or morevehicles approaching the AV, motions or actions in response toencountering an intersection; execution of a merge into traffic in anadjacent lane or area of traffic; detection of need to clean one or moresensor and the cleaning of the appropriate sensor; and the like. Otherfeatures of an autonomous vehicle may include those actions or featureswhich are needed for any type of maneuvering, including that needed toaccomplish the features or actions that are reactionary, listed above.Such features, which may be considered supporting features, may include:the ability to maintain an appropriate following distance; the abilityto turn right and left with appropriate signaling and motion, and thelike. These supporting features, as well as the reactionary featureslisted above, may include controlling or altering the steering, enginepower output, brakes, or other vehicle control subsystems 146.

FIG. 4 illustrates an example flowchart of operations performed by anautonomous vehicle. Operation 402 includes determining, by a computerlocated in an autonomous vehicle, that an emergency vehicle is locatedon a road on which the autonomous vehicle is being driven. Operation 404includes obtaining, in response to the determining, a driving relatedoperation of the autonomous vehicle to operate around the emergencyvehicle. Operation 406 includes causing the autonomous vehicle to driveaccording to the driving related operation by sending instructionsrelated to the driving related operation to one or more devices in theautonomous vehicle.

In some embodiments, the obtaining the driving related operation and thecausing the autonomous vehicle to drive according to the driving relatedoperation is performed in response to: (1) the determining that theemergency vehicle is located on the road, and (2) determining that theemergency vehicle is located within a pre-determined distance of alocation of the autonomous vehicle. In some embodiments, the obtainingthe driving related operation and the causing the autonomous vehicle todrive according to the driving related operation is performed inresponse to: (1) the determining that the emergency vehicle is locatedon the road, (2) determining that the emergency vehicle is locatedwithin a pre-determined distance of a location of the autonomousvehicle, and (3) determining that the emergency vehicle is operating asiren. In some embodiments, the obtaining the driving related operationincludes: performing a first determination that a current speed of theautonomous vehicle is greater than a threshold value, and performing asecond determination, in response to the first determination, of anamount of braking force to apply that reduces the current speed of theautonomous vehicle to less than or equal to the threshold value; and theinstructions causing the autonomous vehicle to drive according to thedriving related operation include a first instruction that cause theautonomous vehicle to apply the amount of braking force.

In some embodiments, the second determination is performed in responseto the first determination and in response to a third determination thatthe autonomous vehicle is driving on a lane on the road immediatelyadjacent to an area or another lane where the emergency vehicle islocated. In some embodiments, the obtaining the driving relatedoperation includes: performing a first determination that the autonomousvehicle is driving on the road on a first lane having one side that isimmediately adjacent to an area or another lane where the emergencyvehicle is located, performing a second determination that the roadincludes a second lane immediately adjacent to the first lane and onanother side opposite to the one side, and performing a thirddetermination, in response to the first determination and the seconddetermination, of an amount of steering to apply to steer the autonomousvehicle from the first lane to the second lane; the instructions causingthe autonomous vehicle to drive according to the driving relatedoperation include a first instruction that cause the autonomous vehicleto apply the amount of steering to steer the autonomous vehicle from thefirst lane to the second lane.

In some embodiments, a system for autonomous vehicle operation includesan autonomous vehicle comprising a computer that includes a processorconfigured to implement a method, comprising: determine that anemergency vehicle is located on a road on which the autonomous vehicleis being driven; obtain, in response to the determine, a driving relatedoperation of the autonomous vehicle to operate around the emergencyvehicle; and cause the autonomous vehicle to drive according to thedriving related operation by the processor configured to sendinstructions related to the driving related operation to one or moredevices in the autonomous vehicle.

In some embodiments, the processor of the computer is further configuredto: send, to a second computer located outside of the autonomousvehicle, information related to the emergency vehicle, where theinformation includes a position information that indicates a position ofthe emergency vehicle, and where the information indicates that theemergency vehicle is located within a first pre-determined distance of alocation of the autonomous vehicle; and where the second computercomprises a second processor configured to implement a second method,comprising: receive the information related to the emergency vehicle; inresponse to a determination that the second autonomous vehicle islocated within a second pre-determined distance of a position of theemergency vehicle and that the second autonomous vehicle is drivingtowards the emergency vehicle: send the information related to theemergency vehicle to a second autonomous vehicle, where the secondpre-determined distance is greater than or equal to the firstpre-determined distance.

In some embodiments, the obtain the driving related operation isperformed by the processor configured to: receive a traffic informationof an area towards which the autonomous vehicle is driving; determine,based on the traffic information, that the area has a high trafficlevel; determine an alternate route to drive the autonomous vehiclearound the area having the high traffic level; and determine theinstructions for any one or more of a steering system, brakes, ortransmission to cause the autonomous vehicle to drive on the alternateroute. In some embodiments, the instructions for the driving relatedoperation include a first instruction to apply brakes and a secondinstruction steer the autonomous vehicle from a lane on the road onwhich the autonomous vehicle is being driven and to an area along a sideof the road. In some embodiments, the emergency vehicle includes a lawenforcement vehicle, an ambulance, a fire engine, or a fire truck. Insome embodiments, the emergency vehicle includes a stationary objectthat includes a traffic cone, a flare, or a traffic related signage.

In some embodiments, a non-transitory computer readable program storagemedium having code stored thereon, the code, when executed by aprocessor, causing the processor to implement a method comprising:determining, by a computer located in an autonomous vehicle, that anemergency vehicle is located on a road on which the autonomous vehicleis being driven; obtaining, in response to the determining, a drivingrelated operation of the autonomous vehicle to operate around theemergency vehicle; and causing the autonomous vehicle to drive accordingto the driving related operation by sending instructions related to thedriving related operation to one or more devices in the autonomousvehicle.

In some embodiments, the obtaining the driving related operation and thecausing the autonomous vehicle to drive according to the driving relatedoperation is performed in response to: (1) determining that theemergency vehicle is driving towards the autonomous vehicle, (2)determining that the emergency vehicle is located within apre-determined distance of a location of the autonomous vehicle, and (3)determining that the emergency vehicle is operating a siren. In someembodiments, the emergency vehicle is determined to be driving towardsthe autonomous vehicle upon determining that a first number of pixelsassociated with the emergency vehicle in a first image is less than asecond number of pixels associated with the emergency vehicle in asecond image, the first image and the second image are obtained by acamera on the autonomous vehicle, and the first image is obtained at afirst time that is earlier in time than a second time when the secondimage is obtained. In some embodiments, the emergency vehicle isdetermined to be driving towards the autonomous vehicle upon determiningthat a first sound level associated with the siren is less than a secondsound level associated with the siren, the first sound level and thesecond sound level are obtained by a microphone on the autonomousvehicle, and the first sound level is obtained at a first time that isearlier in time than a second time when the second sound level isobtained.

In some embodiments, the driving related operation is obtained based ona location of the autonomous vehicle and based on a rule associated withthe emergency vehicle of a state or of an area in which the autonomousvehicle is located. In some embodiments, the driving related operationis obtained using a pre-configured set of rules stored in the computerfor a plurality of states or areas, each rule is associated with atleast one state or at least one area and a corresponding driving relatedrule associated with at least one emergency vehicle, and the pluralityof states or areas include the state or the area in which the autonomousvehicle is located. In some embodiments, the method further includessending, to a second computer located outside of the autonomous vehicle,information related to the emergency vehicle; and where the obtainingthe driving related operation includes receiving, from the secondcomputer, the instructions related to the driving related operation. Insome embodiments, the information is sent to the second computer inresponse to determining that one or more attributes of the emergencyvehicle cannot be determined by the computer in the autonomous vehicle.

In example system as described in this patent document comprises anautonomous vehicle (AV) that comprises an authority detection subsystem,an in-vehicle control computer, and an autonomous control subsystem. Thein-vehicle control computer comprises a legal authority detectionmodule, and a compliance module. In some embodiments, the authoritydetection subsystem comprises microphones mounted, or located, on anexternal portion of the autonomous vehicle. In some embodiments, themicrophones are mounted such that they are effective both when theautonomous vehicle is at rest, as well as when the autonomous vehicle ismoving at normal driving speeds.

An example method for operating an autonomous vehicle comprisesdetecting, by an authority detection subsystem of the autonomousvehicle, the presence of an authority; determining, by a legal authoritydetection module of the autonomous vehicle, directions given by theauthority; determining, by the compliance module of the autonomousvehicle, that a change of trajectory or route for the autonomous vehicleis required to comply with the directions given by the authority; andmodifying the trajectory or route of the autonomous vehicle to complywith the instructions given by the authority.

In this document the term “exemplary” is used to mean “an example of”and, unless otherwise stated, does not imply an ideal or a preferredembodiment.

Some of the embodiments described herein are described in the generalcontext of methods or processes, which may be implemented in oneembodiment by a computer program product, embodied in acomputer-readable medium, including computer-executable instructions,such as program code, executed by computers in networked environments. Acomputer-readable medium may include removable and non-removable storagedevices including, but not limited to, Read Only Memory (ROM), RandomAccess Memory (RAM), compact discs (CDs), digital versatile discs (DVD),etc. Therefore, the computer-readable media can include a non-transitorystorage media. Generally, program modules may include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Computer-or processor-executable instructions, associated data structures, andprogram modules represent examples of program code for executing stepsof the methods disclosed herein. The particular sequence of suchexecutable instructions or associated data structures representsexamples of corresponding acts for implementing the functions describedin such steps or processes.

Some of the disclosed embodiments can be implemented as devices ormodules using hardware circuits, software, or combinations thereof. Forexample, a hardware circuit implementation can include discrete analogand/or digital components that are, for example, integrated as part of aprinted circuit board. Alternatively, or additionally, the disclosedcomponents or modules can be implemented as an Application SpecificIntegrated Circuit (ASIC) and/or as a Field Programmable Gate Array(FPGA) device. Some implementations may additionally or alternativelyinclude a digital signal processor (DSP) that is a specializedmicroprocessor with an architecture optimized for the operational needsof digital signal processing associated with the disclosedfunctionalities of this application. Similarly, the various componentsor sub-components within each module may be implemented in software,hardware or firmware. The connectivity between the modules and/orcomponents within the modules may be provided using any one of theconnectivity methods and media that is known in the art, including, butnot limited to, communications over the Internet, wired, or wirelessnetworks using the appropriate protocols.

While this document contains many specifics, these should not beconstrued as limitations on the scope of an invention that is claimed orof what may be claimed, but rather as descriptions of features specificto particular embodiments. Certain features that are described in thisdocument in the context of separate embodiments can also be implementedin combination in a single embodiment. Conversely, various features thatare described in the context of a single embodiment can also beimplemented in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or a variation of a sub-combination. Similarly, whileoperations are depicted in the drawings in a particular order, thisshould not be understood as requiring that such operations be performedin the particular order shown or in sequential order, or that allillustrated operations be performed, to achieve desirable results.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this disclosure.

What is claimed is:
 1. A method of autonomous driving operation,comprising: determining, by a computer located in an autonomous vehicle,that an emergency vehicle is located on a road on which the autonomousvehicle is being driven; obtaining, in response to the determining, adriving related operation of the autonomous vehicle to operate aroundthe emergency vehicle; and causing the autonomous vehicle to driveaccording to the driving related operation by sending instructionsrelated to the driving related operation to one or more devices in theautonomous vehicle.
 2. The method of claim 1, wherein the obtaining thedriving related operation and the causing the autonomous vehicle todrive according to the driving related operation is performed inresponse to: (1) the determining that the emergency vehicle is locatedon the road, and (2) determining that the emergency vehicle is locatedwithin a pre-determined distance of a location of the autonomousvehicle.
 3. The method of claim 1, wherein the obtaining the drivingrelated operation and the causing the autonomous vehicle to driveaccording to the driving related operation is performed in response to:(1) the determining that the emergency vehicle is located on the road,(2) determining that the emergency vehicle is located within apre-determined distance of a location of the autonomous vehicle, and (3)determining that the emergency vehicle is operating a siren.
 4. Themethod of claim 1, wherein the obtaining the driving related operationincludes: performing a first determination that a current speed of theautonomous vehicle is greater than a threshold value, and performing asecond determination, in response to the first determination, of anamount of braking force to apply that reduces the current speed of theautonomous vehicle to less than or equal to the threshold value; andwherein the instructions causing the autonomous vehicle to driveaccording to the driving related operation include a first instructionthat cause the autonomous vehicle to apply the amount of braking force.5. The method of claim 4, wherein the second determination is performedin response to the first determination and in response to a thirddetermination that the autonomous vehicle is driving on a lane on theroad immediately adjacent to an area or another lane where the emergencyvehicle is located.
 6. The method of claim 1, wherein the obtaining thedriving related operation includes: performing a first determinationthat the autonomous vehicle is driving on the road on a first lanehaving one side that is immediately adjacent to an area or another lanewhere the emergency vehicle is located, performing a seconddetermination that the road includes a second lane immediately adjacentto the first lane and on another side opposite to the one side, andperforming a third determination, in response to the first determinationand the second determination, of an amount of steering to apply to steerthe autonomous vehicle from the first lane to the second lane; whereinthe instructions causing the autonomous vehicle to drive according tothe driving related operation include a first instruction that cause theautonomous vehicle to apply the amount of steering to steer theautonomous vehicle from the first lane to the second lane.
 7. A systemfor autonomous vehicle operation, the system comprising an autonomousvehicle comprising a computer that includes a processor configured toimplement a method, comprising: determine that an emergency vehicle islocated on a road on which the autonomous vehicle is being driven;obtain, in response to the determine, a driving related operation of theautonomous vehicle to operate around the emergency vehicle; and causethe autonomous vehicle to drive according to the driving relatedoperation by the processor configured to send instructions related tothe driving related operation to one or more devices in the autonomousvehicle.
 8. The system of claim 7, wherein the processor of the computeris further configured to: send, to a second computer located outside ofthe autonomous vehicle, information related to the emergency vehicle,wherein the information includes a position information that indicates aposition of the emergency vehicle, and wherein the information indicatesthat the emergency vehicle is located within a first pre-determineddistance of a location of the autonomous vehicle; and wherein the secondcomputer comprises a second processor configured to implement a secondmethod, comprising: receive the information related to the emergencyvehicle; in response to a determination that the second autonomousvehicle is located within a second pre-determined distance of a positionof the emergency vehicle and that the second autonomous vehicle isdriving towards the emergency vehicle: send the information related tothe emergency vehicle to a second autonomous vehicle, wherein the secondpre-determined distance is greater than or equal to the firstpre-determined distance.
 9. The system of claim 7, wherein the obtainthe driving related operation is performed by the processor configuredto: receive a traffic information of an area towards which theautonomous vehicle is driving; determine, based on the trafficinformation, that the area has a high traffic level; determine analternate route to drive the autonomous vehicle around the area havingthe high traffic level; and determine the instructions for any one ormore of a steering system, brakes, or transmission to cause theautonomous vehicle to drive on the alternate route.
 10. The system ofclaim 7, wherein the instructions for the driving related operationinclude a first instruction to apply brakes and a second instructionsteer the autonomous vehicle from a lane on the road on which theautonomous vehicle is being driven and to an area along a side of theroad.
 11. The system of claim 7, wherein the emergency vehicle includesa law enforcement vehicle, an ambulance, a fire engine, or a fire truck.12. The system of claim 7, wherein the emergency vehicle includes astationary object that includes a traffic cone, a flare, or a trafficrelated signage.
 13. A non-transitory computer readable program storagemedium having code stored thereon, the code, when executed by aprocessor, causing the processor to implement a method comprising:determining, by a computer located in an autonomous vehicle, that anemergency vehicle is located on a road on which the autonomous vehicleis being driven; obtaining, in response to the determining, a drivingrelated operation of the autonomous vehicle to operate around theemergency vehicle; and causing the autonomous vehicle to drive accordingto the driving related operation by sending instructions related to thedriving related operation to one or more devices in the autonomousvehicle.
 14. The non-transitory computer readable program storage mediumof claim 13, wherein the obtaining the driving related operation and thecausing the autonomous vehicle to drive according to the driving relatedoperation is performed in response to: (1) determining that theemergency vehicle is driving towards the autonomous vehicle, (2)determining that the emergency vehicle is located within apre-determined distance of a location of the autonomous vehicle, and (3)determining that the emergency vehicle is operating a siren.
 15. Thenon-transitory computer readable program storage medium of claim 14,wherein the emergency vehicle is determined to be driving towards theautonomous vehicle upon determining that a first number of pixelsassociated with the emergency vehicle in a first image is less than asecond number of pixels associated with the emergency vehicle in asecond image, wherein the first image and the second image are obtainedby a camera on the autonomous vehicle, and wherein the first image isobtained at a first time that is earlier in time than a second time whenthe second image is obtained.
 16. The non-transitory computer readableprogram storage medium of claim 14, wherein the emergency vehicle isdetermined to be driving towards the autonomous vehicle upon determiningthat a first sound level associated with the siren is less than a secondsound level associated with the siren, wherein the first sound level andthe second sound level are obtained by a microphone on the autonomousvehicle, and wherein the first sound level is obtained at a first timethat is earlier in time than a second time when the second sound levelis obtained.
 17. The non-transitory computer readable program storagemedium of claim 13, wherein the driving related operation is obtainedbased on a location of the autonomous vehicle and based on a ruleassociated with the emergency vehicle of a state or of an area in whichthe autonomous vehicle is located.
 18. The non-transitory computerreadable program storage medium of claim 17, wherein the driving relatedoperation is obtained using a pre-configured set of rules stored in thecomputer for a plurality of states or areas, wherein each rule isassociated with at least one state or at least one area and acorresponding driving related rule associated with at least oneemergency vehicle, and wherein the plurality of states or areas includethe state or the area in which the autonomous vehicle is located. 19.The non-transitory computer readable program storage medium of claim 13,further comprising: sending, to a second computer located outside of theautonomous vehicle, information related to the emergency vehicle; andwherein the obtaining the driving related operation includes receiving,from the second computer, the instructions related to the drivingrelated operation.
 20. The non-transitory computer readable programstorage medium of claim 19, wherein the information is sent to thesecond computer in response to determining that one or more attributesof the emergency vehicle cannot be determined by the computer in theautonomous vehicle.